feat(zarr-metadata)!: write store bytes through a JSONWriter - #429

… each spec claim (zarr-developers#4605)

* docs(zarr-metadata): split the landing page into guide pages and link each spec claim

Shrink index.md to a short pitch, one example and a reader-by-reader list
of where to go. Move the rest into guide pages: models and readers, what
the readers check, consolidated metadata, repairing known writer bugs,
writing an extension, JSON Schema and Pydantic. Each claim backed by a
Zarr spec links to the spec section that makes it; the consolidated
metadata page says what the specs do and do not settle and links the
zarr-specs PRs and issues and the original zarr-python implementation.
Trim the README so it no longer mirrors the landing page, and say
"context" where the pages said "scope".

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): number the guide-pages fragment after its fork PR

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): say "member" for a JSON name/value pair in the guide pages

The rename branch keeps "member" for a name/value pair of a metadata
object, so the new pages follow: "a top-level member that the spec does
not define is an extra field". "Extra field" and "extension field" stay,
after the names extra_fields and ZarrV3ExtensionField.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): say "member" for a JSON name/value pair on every page

Sweep the parsing-validation and context pages, which said "field" for a
name/value pair of a metadata object, and state the one exception on the
models page: "extra field" follows the spec's "additional fields" and
the names extra_fields, ExtraFieldDefinition and must_understand_fields;
"field" otherwise means a field of a Python class, or a part of a
structured data type.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): say "member" in the two must_understand_fields sentences

The models page said "each unclaimed field" and "any such field" where the
landing page and every neighboring sentence say "member". Roborev flagged
the mismatch.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): say why the additional-fields link points at the latest spec

v3.1.0 has no "Additional fields" heading; PR 373 added it after the
release, so the link cannot be pinned. Say so beside the link.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): split the additional-fields sentence and date its claim

Roborev found the three-clause sentence hard to parse and its "no tagged
release yet" unanchored in time. Two sentences now, anchored to v3.1.0,
with a note to pin the link once a release carries the heading.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): fix six wording and example issues roborev found on the guide pages

The pydantic example now builds the manifest with model_validate, so it
type-checks as the next sentence claims. The consolidated page says three
other readers take a context, not two. The equality section quotes the v2
spec on a missing .zattrs in two sentences. The two numbered Sphinx
anchors are replaced by line ranges in the spec source at the v3.1.0 tag.
The landing page says once that the library does no chunk or store I/O.
The extensions page example and the zarr_metadata.definition docstring
example are pinned to each other by a test.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(zarr-metadata): make the example-drift test say what it could not find

Roborev asked for diagnostics: the test now asserts there is exactly one
extensions-session block and exactly one marker line in the docstring,
each with a message, and compares line lists so a mismatch points at the
differing line.

Assisted-by: ClaudeCode:claude-fable-5-1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(zarr-metadata): build the extension example with union and say "context" throughout

On main a Context has no update method, so the acme.lz4 example on the
extensions page now adds its definition with union, as the definition
module docstring does. Two leftover "scope"s in the parsing page say
"context", and the models page says "member" for an extra field. The
example-drift test imports at module level.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(zarr-metadata): number the guide-pages fragment after upstream PR 4605

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(zarr-metadata): fence the module docstring examples as python so the API pages highlight them

The examples in the definition, json_schema, pydantic and typed_json
module docstrings were indented code blocks. An indented block names no
language and pymdownx.highlight does not guess one, so the API reference
rendered them as plain text. Each is now a ```python fence, and the
example-drift test reads the definition docstring's fence.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(zarr-metadata): write the guide pages in short sentences and lists

Long comma-chained sentences on the guide pages, README and landing page
are split into short sentences, and enumerations become bulleted or
numbered lists. models, validation-rules, consolidated-metadata, repair,
extensions, json-schema and pydantic are restructured; context and
parsing-validation keep their narrative voice and only have their long
sentences split. No link, claim or code block changes.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

`to_key_value` takes a writer instead of an `indent` keyword. A
`JSONWriter` owns how a document becomes JSON bytes: it holds the
settings (`indent`, `sort_keys`, `ensure_ascii`, `trailing_newline`)
and its `encode` method does the writing, so the separators, the bare
non-finite tokens and the surrogate escapes are no longer split off in
a private helper. `to_key_value` also accepts a mapping of the settings,
`JSONWriterSchema`, so callers need not import the class:
`to_key_value({"indent": 2})`.

The defaults are those of `json.dumps`, so a bare `to_key_value()` writes
the same bytes as before. A setting `json.dumps` would misread raises,
and so does a mapping key that is not a setting.

Rebased onto main as one commit, replacing the JSONWriterConfig design.

Assisted-by: ClaudeCode:claude-opus-5-5

…velopers#4521)

* Remove zarr.tree and the Array.compressor properties

Both were deprecated in 3.0.0 and were scheduled for removal in 3.2.0
by zarr-developers#3325. The rest of that PR shipped in 3.2.0 through zarr-developers#3900-zarr-developers#3903.

Removed:
- zarr.tree, zarr.api.synchronous.tree, zarr.api.asynchronous.tree.
  Use Group.tree / AsyncGroup.tree.
- Array.compressor and AsyncArray.compressor. Use Array.compressors.

The warning-only test for zarr.tree is gone, and the v2 compressor
assertions in tests/test_v2.py now use `compressors`.

The zarr.storage.default_compressor shim from the same zarr-developers#3325 list is
left alone; zarr-developers#4505 removes it.

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

…arr-developers#4504)

* fix: reset the codec-pipeline thread pool after fork

A forked child inherits codec_pipeline._pool pointing at threads that
do not exist in the child, plus the parent's idle-semaphore permits.
Submissions up to the permit count are consumed without spawning a
thread, so small FusedCodecPipeline reads hang forever.

Add an after_in_child hook that drops the inherited pool, size, and
lock. The lock is recreated because it may have been held by a thread
that no longer exists in the child.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: add changelog entry for zarr-developers#4483

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(codec_pipeline): pin pool size in the fork regression test

Move the zarr-developers#4478 regression test next to the other FusedCodecPipeline
pool tests and pin `codec_pipeline.max_workers` to 4 through the shared
`_FUSED_POOL_CONFIG`. With the default, a 1-CPU host resolves
`max_workers` to 1, never creates the pool, and passes the test without
exercising the bug. The test now asserts that the parent created the
pool before forking, and it runs on macOS, where the fork start method
is available.

Assisted-by: ClaudeCode:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: barlowa124 <barlowa124@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat: read only the needed rows of uncompressed chunks

BytesCodec now implements partial decoding. An uncompressed chunk is
stored in C order, so each row along its first axis is a contiguous run
of bytes; for a selection that does not touch every row, the rows from
the first to the last one it touches are fetched with a single range
request and the selection is applied to them. The pipeline already
uses partial decoding when the array-to-bytes codec supports it and
there are no array-to-array or bytes-to-bytes codecs, so this applies to
uncompressed arrays only. Reading 100 rows of a single-chunk 20 MB
array now fetches the bytes of those rows instead of the whole chunk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix boolean masks and stores that ignore ranges in BytesCodec partial reads

A boolean mask on the first axis, as from arr.oindex[mask], was trimmed to
start at the first selected row but not to end at the last, so its length
no longer matched the rows fetched and numpy raised an IndexError. The row
window and the shifted selection are now computed together in _row_window,
which trims the mask at both ends. This also gives mypy the narrowing it
needs.

FsspecStore over HTTP returns the whole object when a server ignores the
Range header. The partial read then failed to reshape a chunk that the full
read would have decoded, so a read that worked before this branch raised an
error. A response longer than the requested range is now sliced to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add sync partial decode to BytesCodec and check the length of ranged reads

Add _decode_partial_sync so FusedCodecPipeline reads only the needed rows
of an uncompressed chunk, sharing the decode step with the async method.

Decide what a store sent for a range request by its exact length: the
requested range, the whole chunk, or everything from the range start.
Any other length raises instead of returning wrong data.

Run the partial read tests on both codec pipelines and add a changelog
entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

…4522)

* Deprecate the ignored Zarr-Python 2 keyword arguments

`synchronizer`, `chunk_store`, `cache_metadata`, `cache_attrs`,
`object_codec`, `read_only` and `meta_array` are accepted by `create`,
`group` and `open_group` (and by everything that wraps them, such as
`zeros`) but have never done anything in Zarr-Python 3. Passing one
used to emit a `ZarrRuntimeWarning` saying "not yet implemented", which
read as a promise to implement it.

Emit a `ZarrDeprecationWarning` instead, saying the argument is ignored
and will be removed in a release after April 2027, and say the same in
the docstrings and the v3 migration guide. Calls that do not pass these
arguments are unaffected.

First step of the plan for zarr-developers#3976.

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

…ers#4612)

Python shows a DeprecationWarning by default only when it is attributed
to __main__, so warnings that pointed at zarr internals were hidden
from users running scripts:

- `ArrayV2Metadata.chunk_grid` is a cached_property, which adds a frame;
  stacklevel 2 pointed at functools. Now 3.
- Setting `zarr.storage.default_compressor` used stacklevel 1, which
  pointed at zarr itself. Now 2.